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ABSTRACT 

Past rerearch has shown that prior knowledge has a 
main influence on Ihe learning of students and is a better predictor 
of study success than other variables in distance education. It has 
also become clear than the use of assessment techniques as 
instruments for gathering information on study progress and for 
directing study guidance should be increased, even though 
implementing such research results in educational practice is not 
easy. A first step is to answer the question of whether students 
themselves support the "overall assessment prophecy" and whether they 
do intend to use the instruments. These questions were investigated 
through questionnaire completed bv 1,159 students of the Open 
University of the Netherlands. Results show that students largely 
agree with the prophecy and are highly prepared to use the proposed 
assessment instruments, which are prior knowledge state tests and 
progress tests. Students want these instruments for different 
purposes, such as knowing their own strengths and weaknesses, 
directing their studies, and making the most r^pid progress possible. 
Differences are found between beginning and advanced students and 
among study domains. Twelve figures and 13 tables present study 
findings. (Contains 60 references.) (Author/SLD) 
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Abstract 

Past research has shown that prior knowledge has a main influence on the learning of students and is a 
better predictor for study success than other variables^ also in distance education. Moreover it has become 
clear that the use of assessment techniques as instruments for gathering information on study progress 
and for direaing study guidance should be increased. However, implementing such research results in 
educational practice is not that easy. A first step is a clear answer to the question if students themselves 
do support the 'overall assessment prophecy' and if they do intend clearly to use the instruments. The 
present study investigates this matter using a questionnaire for a sample of Open university students. The 
results show that students largely agree with the prophecy and show a high preparedness to use the 
proposed assessment instruments, being prior knowledge state tests and progress tests. Remarkable is the 
finding that students do want to use the instruments for strongly different purposes (knowing their 
strengths and v'eaknesses; directing their study guidance; attaining the fastest study progress as possible) 
and that there are differences between domains and between beginning and advanced students. 
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1 Introduction and theoretical background 

1.1 Earlier research on prior knowledge and progress 

Earliec research (Dochy, 1992; Dochy & Valcke, 1993; Dochy & Alexander, 1994) shows that prior 
knowledge of students is a determining factor in the learning process of students in higher education, both 
in a regular university as in a distance teaching university. A methodology for using Prior Knowledge State 
Tests was introduced. This methodology took as a starting point the finding that the structure is one of the 
most important aspects in taking the students* prior knowledge into account. 

Starting from the overall-assessment prophecy, i.e. that assessment becomes an integral part of instruction, 
we have investigated the usefulness of introducing tests for assessing the prior knowledge state of students 
at the beginning of a their study. A similar procedure for assessing the students* knowledge states during the 
study period has been called progress tests (Dochy & Bouwens, 1992; Dochy, 1992). 



1.2 A conceptual framework for studying knowledge acquisition and the structure of knowledge 

Starting from a European-American conceptual framework (Dochy & Alexander, 1994), the prior knowledge 
state (PKS) is defined as the knowledge state comprising existing declarative and procedural knowledge which 
meets the following conditions: it is present before the implementation of a particular learning task, it is 
available or able to be recalled or reconstructed; it is relevant for the achievement of the objectives of the 
learning task, it is organized in structured schemata, it is to a certain degree transferable or applicable to other 
learning tasks, it is dynamic in nature. For a clear definition of all terms used in relation to knowledge 
acquisition we refer to Dochy and Alexander (1994). 

The structure of knowledge issue has been dealt with by a variety of theoretical disciplines: cognitive 
psychology, educational psychology, artificial intelligence, etc. From a pragmatic point of view, the issue has 
also been of prime importance in applied sciences like instructional psychology, curriculum development 
theories and psychometrics. An early, cognitive-theoretical approach appears in the work of Ausubel (1968) 
who argues that new knowledge is only acquired to the extent that it is meaningfully related to existing 
knowledge. Ausubel maintains that knowledge is organized primarily in a hierarchical fashion, which implies 
that m'astery of higher knowledge levels assumes mastery of all lower knowledge levels. Additionally, Ausubel 
advances that the various pieces of information integrated within a particular knowledge structure are highly 
interrelated. Thus, the more structured prior knowledge, the more flexible and easy the acquisition of new 
knowledge becomes. 

AusubePs conceptualization of learning as the assimilation into prior knowledge, is echoed and extended 
in Mayer*s (1979) schema theory. New knowledge is - according to Mayer - assimilated into a hierarchy of 
progressively more specific content within the learner*s knowledge base. Thus, the basic learning process can 
be described as the assimilation of new knowledge within hierarchically ordered schemata. 

Another benchmark is set by the *elaboration theory*. According to this theory, multiple access avenues 
become available to the learner by the activation of alternate relational paths. This theory also depended on 
the assumption that knowledge acquisition is facilitated to the extent that information is organized in a 
hierarchically integrated mode (Reigeluth & Stein, 1983). 

From a rather pragmatic, e.g. instructional-psychological, point of view the stracture-of-knowledge paradigm 
needs to be investigated in detail in order to find more efficient ways for using instructional technology. 
Research into ways of utilizing knowledge states (Dochy, 1992) indicate e.g. that the different components 
of the PKS should be taken into accpunt (e.g. at the subject-matter level) and that components of the PKS 
along other dimensions (e.g. educational-psychological dimension) could be helpful in educational settings for 
diagnosis and as a basis for intervention. 

An important conclusion from our former research indicates that i : looks promising to analyze in more detail 
the complex of components of the Knowledge States (KS). A first attempt in this direction focused on the 
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Structure of the PKS along the content dimension (Dochy, 1992). Note that we use the term 'component' 
to refer to these subparts of the prior knowledge state that relate to a specific parameter along a knowledge 
profile dimension, as our knowledge state tests did refer to at the content dimension. 

In order to clearly understand this statement, it is necessary to stress that three important concepts are 
introduced in the former paragraph: the KS, components of the KS and a complex of components. These 
three concepts refer to the value attached to the specific 'structure' of the KS. Our earlier research revealed 
that such a structvire could be indicated in prior knowledge, e.g. along the content dimension. We did for 
instance discriminate between the optimal requisite prior knowledge state and the mathematics prior 
knowledge state. But it was also suggested that the differentiation of components of the KS along other 
dimensions could be helpful to diagnose and support educational practice. For a c^ear distinction between 
'parameter' and 'dimension' we refer to Figure 1. 



1.3 Knowledge profiles 

As such, the concept of 'knowledge profiles' is not found in literature. Only 'student profiles' (Wolf, et al., 
1991) and 'cognitive profile' (Lctteri et al., 1982) have some similarity in meaning. 

Our use of the concept 'profile' is derived from the practice, common in educational research, of plotting 
as a graph (or profile) the raw or standardized scores of a group or individual on certain parameters (Keeves, 
1988). In analyzing research data, comparisons are made between persons or groups of persons in terms of 
a set of measurements about related aspects. For each person or group a profile can be obtained by 
combining the results of the set of parameters. The comparison between individual profiles is known by the 
generic term 'profile analysis'. 
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Figure 1: Example of a knowledge profile 

Figure 1 shows the relationships between some key concepts. A 'dimension' is the basis to construct a 
knowledge profile. Each dimension represents an approach towards the structure of knowledge since it 
introduces a related set of KS components. These components are named parameters. 

From an instructional-psychological point of view, knowledge profiles can give practical indications of 
student achievement and learning, thereby making it possible to direct the learning prc.ess. In a recent 
overview of student assessment. Wolf et al. (1991) advocated this approach. Accordmg to these authors, there 
is a need for new educational psychometrics in line with the much changed perceptions of educational 
achievement. The educational world must come to terms with new premises, multiple paths towards the 
prior knowledge state, more developmentally-oriented assessments and the students entering school with 
widely varying backgrounds. For our part, we take account of these changes by trying to identify the compo- 
nents of the prior knowledge state, by implementing prior knowledge state tests and by initialisation of plans 
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to use these tests several times a year to monitor students* progress. 

In this context it is necessary to come to an ;*greement on the relevant parameters for describing student 
performance and it is critical to develop ways of looking at 'student profiles*: " unless we develop these kinds 
of differentiated portraits of student performance within a domain, it is difticult to envision student 
assessment ever informing, rather than merely measuring, the educational process" (Wolf et al., 1991). 

Our multi-profile approach generates 'profile analysis' pochy & Valcke, 1993; Valcke & Dochy, 1991), an 
application of multivariate analysis of variance (MANOVA) in which several dependent variables are 
measured on the same scale (Tabachnick & Fidell, 1989). This profile analysis can provide relevant 
information with diagnostic and remedial value. In this way, the profiles help us to identify learning deficits 
to be remediated. 

Data were found that support the relevance and validity of the knowledge profile dimensions. Two 
approaches were to be adopted, based on data gathered during an investigation involving a large sample of 
university students. First, we analyzed the extent to which the parameters along the dimensions give 
information about different components of the PKS. Second, we analyzed the discriminatory power of the 
knowledge profile dimensions to make apparent PKS differences between a variety of student sub-populations 
(Wagemans et al., 1991; Valcke Bc Dochy, 1991; Dochy & Valcke, 1993). 
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2 The scope of the present investigation 

The starting point of this investigation has been a variety of problems observed in the context of higher 
education, e.g. the problem of equal opportimities for all students at the start, the problem of 
multifunctionality of modules in modular education, the problem of sequence in which bourse modules are 
studied, the problem of differing students, the problem of making appropriate use of prior knowledge. Prior 
knowledge is a central issue in relation to each of these specific topics. 

To solve these problems, it is not enough to work out a methodology and practical instruments. Our new 
approach towaroi the analysis of prior knowledge foresees that in situations where there are significant 
differences between the PKS of specific subpopulations, the profile dimensions will be helpful to detect and 
dissect in more detail the strengths and weaknesses of the students involved. This might be a promising 
starting point for differentiated diagnostic and guidance approaches. 

Next to that, a clear view on how students perceive these new instniments and on their future habits will 
be necessary: their needs, their preparedness to \ise, their possible investment, etc. This will be investigated 
in the present study. A sufficient acceptance for the overall assessment prophecy and its practical 
implementation will urge to a new teaching-learning model as presented in Figure 2. 
Then instruction is to be reconsidered, at least in open learning environments, on* the basis of a new 
educational model of the learning process in which the overall assessment takes a central place and tne 
students* prior knowledge state is the starting point. 
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Figure 2: Assessment at the core of the teaching-learning model 
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According to this model, the student starts with stating his learning goals. These relate to a certain part of 



2. LI Effects of prior knowledge 

Earlier cited theories of Axisubel, Mayer, and Reigeluth & Stein are important since they stress the 
'structured', hierarchical nature of prior knowledge and indicate that prior knowledge is an important 
variable in learning. Additional support, especially for the importance of prior knowledge in learning 
(measiu-ed by a Prior Knowlec^e Sute Test), both in the sense of high correlations and highly explained 
variance (20 to 45% in ecologically valid settings and up to 90% in experimental settings), has been provided 
by the investigations of Bloom (1976), Lodewijks (1981), Parkerson et al. (1984), Korkel (1987), Alexander 
et al. (1989), Weinert (1989), Minnaert and Janssen (1992), Dochy (1992) and Valcke and Dochy (1991). From 
these investigations, one can derive that the 'building' comparison is important in learning, which means that 
the student*s knowlec^e state is not only important at the start of his study, but remains an important 
variable during the study (i.e. at the start of each building block, at the beginning of each learning task, etc.). 

Although - as shown earlier - the impact of prior knowledge is often stressed to this extent that all learning 
might even depend on it (Resnick, 1981), also other factors like other student charaaeristics do influence the 
learning process and can interact with the impact of prior knowledge. But, it is not yet clear which personal 
or contextual variables play a significant role in this context (Ferguson-Hessler, 1989). Differentiating between 
groups of students on the basis of such variables does not seem to make much sense for educational praaice. 
Moreover, our research indicates that, if different variables are taken into account, prior knowledge always 
has the strongest general effect and overnUes other variables in descriptive and explanatory models (see also 
Ethington, 1990; Bniinsma & Geurts, 1988; Dochy, 1988), Analysis in terms of the High Knowledge (HK) 
and the Low Knowledge (LK) groups (in the sense of higher education and duration of work experience) 
show a clear distinction between groups (Dochy ic Steenbakkers, 1988). The majority of the HK group 
students found the course easy, whereas the opinions of the LK group students were divided. This assumption 
supports the general results of research into prior knowledge and the fundamental issue of the different Prior 
Knowledge theories, i.e. that students* prior knowledge makes learning easier. 

This can be explained by the influence of the direct effect of prior knowledge on learning. By direct effect 
is understood all effects as found in situations in which prior knowledge was found to perform an influence 
on study results of students. In those studies, the amount of knowledge at which the student has th^ disposal 
of, is directly related to the expected study success (see Niestadt & Dochy, 1992). Certainly there are 
investigations which contradict the general direct effect. The study by Palmer et al. (1979) is such an example. 
However, they conclude that having high school economics may in some cases mislead them into over- 
confidence. 

Besides studies into direct effects of prior knowledge on study resxilts, research has been done into the 
indirect effects on study results, for example effects on study time. Indirect effects come to surface when 
investigating for example interactions between prior knowledge and instruction. Prior knowledge is here 
found to have effects on study results that are not plainly caused by the knowledge possessed. In those 
studies, prior knowledge is found to affect the required time for study, the effort one has to spare, etcetera. 
When we draw our attention to the indirect effect of prior knowledge, then all consequences of prior 
knowledge are paramount, in which prior knowledge appears to be of influence for effects which, in turn, 
directly affect a student^s study results. 

For example* Saunders (1980) found that students who had taken one or more domain-specific high school 
courses knew significantly more at the end of a university principles course, even though they reported 
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working significantly fewer hours on the university course. Other researchers (see Niestadt & Dochy, 1992) 
concluded students with high SAT scores learned more per time unit in any type of intellectual endeavour. 
A conclusion from these studies into indirect effects can be that instruction affects new achievements as it 
interacts with the learner*s prior knowledge state or his familiarity with the subject matter. Tobias (1976) 
described a range of levels of instructional support and embedded support devices which can minimize 
individual differences between students. This proposal seems to find a reasonable basis in educational research. 
The well-known study of Bopello (1984) was already a clear warning to education based on self-study in the 
sense that in providing courses to students with different prior knowledge levels, one should take account 
of these differences in the individuals* materials. 

The facilitating effect of prior knowledge is generally recognized by educational researchers as being the most 
important positive effect on learning. In Dochy (1992) different theories that give an explanation for this 
finding are explained. Nevertheless, we should not forget that not all facilitating effects arc the 'direct' results 
of prior knowledge. For the purpose of this review, we can make a distinction between: 

a. a direct effect of prior knowledge facilitating the learning process and leading to better study 
results; 

b. an indirect effect of prior knowledge, optimizing the clarity of study materials; and 

c. an indirect effect of prior knowledge, optimizing the use of instructional and learning time. 

Further, given the fact that we will look for differences between domains in the present study, it is important 
to stress that it has been convincingly shown that prior knowledge states of students are highly domain- 
specific, assuming that students will learn more effectively in some domains than in others (Alexander, 1992; 
Alexander & Kulikowich, 1991; Dochy, 1992). 

2.1.2 Students' perceptions of their own prior knowledge 

An additional approach to determine the prior knowledge level of the students, is to ask the students 
themselves to estimate their mastery level. In an experiment by Lodewijks (1981), it was shown thai prior 
knowledge could account for 36% of the explained variance in post-test scores. In this instance prior 
knowledge was considered to be the subjective rating of familiarity with the content of the learning task. 
However, when considering the Falchikov and Boud (1989) meta-analysis of 48 student self-assessment studies, 
the researchers came to more differentiated conclusions: most studies found positive effect sizes, indicating 
overrating on the part of the students. It appears that more experienced students in a particular field (that 
is, students with more prior knowledge) are more accurate estimators, whereas students taking introductory 
courses appear to make particularly inaccurate self-assessments. 

In our own study (Wagemans, Valcke & Dochy, 1991), it was shown that the estimation of PKS level 
through self-assessment by the students is not very reliable in comparison with objective prior knowledge 
state tests. The estimations do not reflect the levels of PKS as measured by^the (two different) objective tests 
(Figure 3 and 4). For our purpose, it can be concluded that self-assessment is not useful, at least not for 
introductory courses, as also shown by Falchikov and Boud (1989). Moreover, it was found that students have 
very differing conceptions of 'prior knowledge', which makes a self-rating procedure invalid (Dochy, 1992), 
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PKST1 



Figure 3: Differences between self-assessment levels and real PKS levels (PKSTl) 



PKST 2 



Figure 4: Differences between self-assessment levels and real PKS levels (PKST2) 
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2.2 Assessment procedures 
2.2.1 Final examinations 

Several assessment procedures have recently been operationalised at our university in a distance education 
context. Next to the traditional administration of paper-and-pencil examinations in study centres (including 
multiple-choice format to open-ended questions) on the occasion of three moments a year, the SYS-system 
is used. In the SYS-system, the computer generates a unique examination for each individual student by 
selecting items from an item-bank. This automated evaluation format fits extremely well in the Open 
university philosophy guaranteeing a high degree of openness and flexibility. The student can take a final 
examination any moment he wants to. The necessity to inform the study centre about the intent to take an 
examination two weeks beforehand is the only time-limitation. After solving the questions and problems, 
the student gets feedback about his final score. This score is confirmed about two weeks later after item- 
analysis and test-analysis. 

Information is available about appreciation and the perception of students of the SYS-evaluation format. In 
a 1991-study among 1123 Open university students, attitudes towards the SYS-evaluation system were as 
follows: 43% said they consider the system to be moderately suited, 50% indicated the system was well suited. 
In a comparable 1992-study among Ou-students, the appreciation of this examination format raises up to 62% 
of the students (1992-11). When asked to choose among three examination formats, students favoured the most 
flexible format (SYS-examination). Two consecutive studies revealed the following proportions: 

Appreciation of examination formats 
1991 and 1992 




Figure 5: Appreciation of examination formats 

The data in figure 5 undoubtedly indicate the growing appreciation of the SYS-system. This obvious 
preference for SYS-examinations does not differ between students studying different domains (1991-11 and 
1992-1). Further analysis of the quantitative results reveal that students especially appreciate the SYS- 
format because it allows to adapt the examination-moment to the personal ^tudy plan. 

Apart from evaluative data about the SYS-examination system, there is hardly information available about 
the student opinions in relation to the more traditional examination procedures or the examination 
content, the types of items, etc. in general. Moreover, and related to the topic of this research, we 
observe that nearly all investments in evaluation has been in relation to 'final examinations/tests'. Prior 
knowledge state tests and/or progress tests only recently have been considered as possible topics for a 
substantial investment in educational research and development. 
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2.2.2 Alternative assessment procedures: prior knowledge and progress tests 

Prior knowledge state tests (PKST) related to the content of a course have showed to be highly reliable 
instruments (for economics and mathematics, see Dochy, 1992; for history, see Elen, 1993) with a high 
predictive validity as stated earlier. In his profound investigation into prescription for instructional 
design, Elen (1993) concludes that from his different design parameters (cognitive skills, metacognitive 
knowledge and skills, and prior knowledge), only "prior knowledge has teen shown to significantly affect 
learning outcomes and is to a cenain extent also of determining importance". According to Elen, the 
value of prior knowledge cannot be underestimated. 

Further, from our experiences at the university of Maastricht, a university with a problem-based 
curriculum using progress tests four timeij a year for all students, it was clear that using tests several times 
a year to monitor students' progress was a very useful and informative source, both for students and for 
professors (Dochy & Wijnen, 1987),In a study concerning the use of progress tests, Imbos (1989) 
concluded that progress tests are very useful, even for first year students. He discriminated between Tirst 
year curriculum items* and 'general items* and found that laymen have a chance for success of .14 on first 
year items and are able to answer one third of the questions. Imbos suggests two concrete aspects for 
amelioration of progress tests: (1) ordering the items based on p-values and (2) giving students who score 
vary low a tailored test (selected sample of the same itembank adapted to his prior knowledge level). 
Shavelson, Baxter and Pine (1992) have been investigating science performance for several years, designing 
tests to foster good teaching and learning as well as to monitor student progress. Shavelson et al..(1994) 
were using alternative assessment technologies, consistent with the emei^ing construaivist assumptions 
about learning. One can say that these assumptions are surely correct for our target group, being adult 
students. Interesting feedback on student progress was supported by scoring the assessments so as to 
capture not just the right answer, but also the reasonableness of the procedure that was used to solve the 
problem (Shavelson, 1994). 

Finally, the results of the 1992 study (Dochy, 1992) showed that there was a need for more formative 
evaluation during the study periods. These stipulations have led to work out so-called 'progress tests', 
consisting of items at final examination level. In this way, the progress tests give information on the 
discrepancy the students have to bridge in order to reach the final goals of the course(s). 

Related to the overall-assessment prophecy (Dochy, 1992) and the use of progress tests, clearly an 
innovation can lead to interesting results. Snow (1990) states that new approaches to cognitive assessment 
in education should be directed towards initial states, daily skill and strategy transitions, weekly 
knowledge transitions or monthly progress in a course. Clearly, our earlier mentioned model of progress 
tests fits with Snow's la^t approach. There seems to be no doubt that such an innovative model is useful 
when looking at empirical research. Powell et al. (1990) comes to the following conclusions. Probably less 
than 10 percent of the students demonstrated sufficient progress to complete a multi-course program 
within a reasonable length of time. Moreover, 60 percent of first-time students fail to earn any credits 
within 30 months of oeing admitted. It seenrs clear that the problem cannot be handled by contracting 
students because nearly half of the students are unable (or perhaps unwilling at the moment) to state 
their study program intentions (Powell et al., 1990). This fits with the results of van der Velden et al. 
(1992), finding that only 16 percent of the studenti^ in distance education are interested in the advantages 
of such a study contract. 
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2.2.3 Assessment as an alternative support procedure 

As stated above, other - formal - alternative examination provisions are not available in our university. 
But some 'informal' alternative formats, rather known as support devices, are present. For instance, the 
distance education study materials (mostly text-based) comprise a self-test and vsrpes of 'embedded support 
devices' (ESD) of an evaluative nature. A student can - but is not obliged to - make use of these text- 
embedded provisions and can read the written feedback at the end of each learning unit. 

In a recent study - with a small sample of law students - the actual use of such embedded evaluative 
support provisions has been investigated (Poelmans et al, 1993). The findings show that questions, 
prequestions, tasks and feedback on tasks were used by at least 80% of the students. Moreover, 7/ Yo of 
the students, and in most cases up to 90% of the students, made a 'deep level' use of these provisions. 
Deep level use implies that the students extensively pay attention to the ESD, read them and react on 
them, let the ESD influence their study behaviour, etc. In terms of 'appreciation', the evaluative 
embedded support devices were part of a set of highest appreciated ESD. 

In another 1992-study with Ou-students the evaluative support devices in the instructional materials were 
researched with a larger student population from different study domains (N-1907). This study revealed 
that 84% of the students solve more than 50% of the embedded questions and tasks (Boon et al., 1992). 

These results suggest that students have a need for alternative assessment procedures that fit into their 
study approach, study plan, etc. Also it is suggested that the use of these assessment procedures is 
depending on the accessibility and easiness for daily use of these instruments. But study approaches differ 
and study plans represent a wide variety of strategies used by students. So, despite a rather ^general' 
demand for alternative assessment procedures, there remains the question about the adaptation of the 
alternative procedures to each students* study approach. About this topic, there is hardly information 
available in the context of distance education at the Dutch Open university. In this text we therefore pay 
much attention to such alternative evaluation procedures but link this immediately to the issue of student 
'types'. 

2.3 Students' views on innovative procedures 

Concerning the students* views on our stated problem, a preliminary investigation was carried out by 
Dochy and Steenbakkers (1988) to search for an answer to a number of questions on the views of 
students about prior knowledge and the importance that prior knowledge has on learning. 
In-depth telephone interviews were used and a sample of 120 Open University students were approached 
as respondents, divided among fields such as Economics, Natural Sciences and Social Sciences* This sort of 
interview uses pre-structured questions with open response options and can be regarded as a qualitative 
method of research, by Patton (1980) also as *the standardized open-ended interview*. 
Students were asked if they found the courses that they took related to their work / hobby / experience. 
Of the total research population one third of the students agreed that this was the case, although another 
third of the students thought there was no relation. In this last group there was clearly a majority of the 
so-called low-knowledge group (being students with only a few years work experience and no higher 
education degree). 

Concerning the manner in which new knowledge can best ba acquired, students still assign a dominant 
role in the transfer of knowledge to the school. When this is broken down into the HK and the LK 
groups, it appears that traditional views of the acquisition of knowledge are strongest among those with 
the most formal education. The LK group believes that it can also derive knowledge from self study and 
experience. 

Further, it was found that virtually every student thought that the effect of not having the 'prior 
knowledge* would be that the time required for the course takes longer. Another effect would be that 
where there was sufficient motivation and time, the course could be passed, but it would affect the course 
results i.e., a lower score would be achieved in the examination and it would be easier to fail. 



10 



17 



Alternative Assesiment Procedures 



These conclusions are cr^nsistent with the results of a study by Powell et al. (1991), investigating success 
and persistence at two distance teaching universities using questionnaires. Powell et al. found that the 
educational qualifications on entry of students of both universities were associated with success on the 
first course taken, with students having no post-secondary education performing worse than those with 
higher qualifications. 

The study showed that pass rates of first-time students at both universities were remarkably similar at 36 
percent. Pass rates for students who subsequently registered for more courses increased considerably as 
students progressed through their studies. Moreover, it was found that success on the first course taken is 
highly correlated with re-registration. Only 15 % of students who do not pass their initial course, return 
to take another. 



2.4 Types* of students : a rationale 

As stated above, in this study about prior knowledge state tests and progress tests, much attention is 
being paid to 'types' of students. This interest in 'types' of students is an important topic in the general 
context of distance education: courses are to be multi-functional; 'clients' have very different 
backgrounds/profiles (educational level, professional background, study plan, study speed,...); etc/ 
The fact we concentrate so much on 'types' of students is the result of empirical, theoretical and 
pragmatic considerations: 

Institutional research with Ou-students (Boon et al, 1990) indicated e.g. the importance of 
variables such as educational background to explain differences in attitudes towards aspects of 
distance education. Our own earlier research in relation to prior knowledge research revealed 
that prior knowledge among students differs depending on their prior educational experiences, 
hut only when we group students in two groups, being students with and without an university 
or higher vocational education degree (Dochy, 1992). Research in relation to the actual use of 
study materials reflect the impact of educational level, gender and study plan (Poelmans et al., 
1993). 

At the theoretical level, student variables (independent variables), are incorporated in descriptive 
and explanatory theories or model about learning and teaching at the Open university (e.g. 
Valcke, et al., 1992; Vermunt, 1992). 

From a pragmatic point of view, taking account of student variables als answers a growing need 
to reach more efficient and effective education. There is for instance an urgent demand for more 
cost-effective planning and implementation of support and evaluative provisions; e.g. the number 
of study centres and study centre staff, the number of examinations taken by students,.... A key 
variable in this discussion is the 'type' of student for whom support and evaluative provisions in 
study centres are relevant and needed. The importance attached to the development of better 
intake-procedures is but one outcome of this general concern. 

In this study we operationalise 'types' of students by making use of three variables : 

The level of previous educational experiences; we distinguish two groups of students, a group 
having higher education (higher vocational or university degree) and a group of students which 
highest previous educational experience is of secondary school level. 

The level of success in studying in the distance education setting of the Dutch Open university; 
here we distinguish between four groups of students based on the number of certificates: 0, 1-2, 
3-4, and more than 4 certificates. 

The content doi».Ain according to the students' study plan; the Dutch open university 
distinguishes between the following 7 content domains: law, economics, business, technical, 
natural, cultural and social sciences. 

One can question whether the three operational definitions of the variable 'student type' presented here 
arc relevant and helpful to explain the way students react to evaluation formats. In our study, we do not 
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neglect this question. More specific, we make use of log-linear analysis techniques to evaluate the impact 
of the independent variables when looking for grounds that account for the variance in student responses. 



3. Research Design 

3.1 Research hypotheses 

Six clusters of hypotheses can be put forward taking account of the information obtained from the 
questionnaire (see appendix). The first four sets are related to prior knowledge state tests. The fifth is 
related to progress tests and the sixth set to study contracts: 

a Students have no clear view of their prior knowledge level, 

b Students do not need prior knowledge state tests, 

c The perceived and expected effects of PKST on study variables are limited, 

d The perceived effect of PKST on guidance, support is limited, 
e Progress tests do not answer specific student needs, 

f Study contracts do not answer specific student needs. 

In the results section, each cluster hypothesis will be considered by relating the cluster to specific 
questionnaire items. In this way in-depth results are obtained which are helpful to draw detailed 
conclusions. 

The reader will notice that section f on *study contracts' cannot be considered as an alternative evaluation 
format. As will be explained in section 4.3, we did incorporate this topic due to a animated discassion 
actually found at the Dutch Open university. We consider 'study contracts', in the context of this text, as 
a logical extension to the alternative evaluation formats put forward. In this way we want to know 
whether students are willing to attach formal consequences to the results of new evaluation formats. As 
indicated in section 2.2, the impact of the independent variable 'student type' will be researched in 
relation to each cluster of hypotheses. 

3.2 Research Instrument 

This investigation was designed in cooperation with the department for market research of our 
university. As part of their yearly research with a sample of students, the authors constructed a specific 
set of questions to be incorporated in a 17-page questionnaire (cf. appendix 1). The set of questions about 
prior knowledge state testing and progress tests made up one section in this questionnaire. 
The items in the questionnaire mirror a specific set of research variables put iForward about alternative 
assessment procedures. Each item can be consistently linked to such a variable. The construction of these 
items was not an evident task as will be outlined in the next paragraph. 

If students have no prior experiences with the phenomena researched, how is it possible to get reliable 
and valid responses to question about these phenomena? The designing of a research instrument to get 
relevant information on opinions of students about the alternative assessment procedures researched here, 
is based on the assumption that students are not familiar v/ith these alternative evaluation formats. 
Therefore the design of the instrument takes into account the pitfalls given by Neijens (1987) that can 
appear in the construction of ^n instrument for collecting informed opinions of a population. According 
to Neijens (1987) the four main indicators for the quality of the evaluated opinions are: direction errors, 
consistency errors, halo effects and judgement variation. Most of these can be avoided by providing 
correct information about the phenomena (and linking to their own experience) and by selecting 
carefully the presentation of the information. 

First, in our design process, this solution was operationalised mirroring the alternative evaluation formats 
to aspects of the actual study process and study behaviour of the students. Secondly, in some cases several 
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items were partly overkpping in order to control consistency. Thirdly, by gradually presenting to the 
students other aspects of the alternative evaluative procedures, in terms of inferences on aspects of their 
actxial study behaviour and study process, more reliable information can be obtained about their 
appreciation, attitudes, acceptance of the new Ideas. In this respect it is to be expected that students will 
show some so-called 'orientation behaviour* during the first questions. Example: Time planning of the 
study process towards the final examination is a normal aspect of student study behaviour. If we show to 
the students that progress testing can affect their normal time management, students can take positions 
on a better informed base. This example suggests that in constructing the questionnaire care has been 
taken to present to the students other specific implications, consequences arid characteristics of the 
alternative evaluation formats. Another example is more complex. When testing e.g. the hypothesis 
"Students do not need prior knowledge state tests.", five items in the questionnaire were presented to the 
students, each of them presenting aspects that affect the study behaviour and study process: 

- uncertainty about one mastery when starting a Ou-course 

- time management 

- the way information obtained from prior knowledge profiles is to be used 

- the need for personalised and written feedback 

the lack of feedback in current evaluation practices. 
Despite the care taken in constructing the questionnaire, caution will still be needed when interpreting 
the research and when drawing conclusions. The data obtained from the students can still be biased by 
lack of information about and experience with the alternative evaluation formats. 

As stated above, the questionnaire was carefully constructed by gradually adding specificity to the 
implications of alternative evaluation formats in terms of student behaviour and the study process: 

The questionnaire started with a descriptive text giving a short outline of 'prior knowledge* and 

*prior knowledge state tests*. 

Next, questions were put forward about the students* perception, estimation of their personal 
prior knowledge state/level. 

After this, general questions were put forward about their willingness to take prior knowledge 
state tests. 

In the next questionnaire item, students could react to statements about possible uses that could 
be made with the results of the tests after denving strong and weak points in one*s prior 
knowledge state. 

The following set of questions repeated one by one possible uses of the prior knowledge state 

test results. Here, students could respond in a more elaborated way (5-point scale). Moreover, if 

relevant, an extra variable was added in these questions about the person to make *use* of the 

information obtained: the student him/herself or advisors/experts. 
The subsequent part of the questionnaire consisted of: 

a short introduction on progress tests 

and two questions about this evaluation format 
The questionnaire ended with: 

a short explanation about study contracts 

and questions about the willingness of students to consider taking study contracts. 
Also the questionnaire contained two questions about: 

- feedback on the tests that are embedded in the course materials 

- and the opinion of students about the feedback value of the actual Ou-examinations. 

A first version of the questionnaire was screened by a team of educationalists and adapted following their 
comments. A second version was screened by the staff of the "Onderzoek & Evaluatie" department of 
the Dutch Open university. After revision, the final versi9n was drawn up and incorporated in the larger 
questionnaire. 

3.3 Research Population 

An a-select sample of 2000 Ou-students was taken from the student population that subscribed for a 
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course in the period September 1990 • September 1991 (N-35.313). This sample was stratified according 
to the variable 'success* in passing course modules. Of this sample, 1139 students responded by returning 
the questionnaire (58%). A weighting of the respondents was necessary taking the variables 'number of 
courses taken* and 'number of modules passed*. Every course exists of a certain number of modules 
depending on the study load of the course. Therefore, the modules give a more precise indication of a 
students* studyprogress. The results of the weighting procedure was a weighted number of 1907 students, 
each having passed the same amount of modules in a certain domain. As a matter of fact, in the following 
paragraphs n will always refer to this weighted number. 

3.4 Procedure and data analysis 

As stated above, the questions in relation to the alternative examination formats were part of a larger 
questionnaire. Early in 1992, the questionnaire was sent to the students. The extraction of the research 
data from the questionnaire forms was done by staff of the department for market research. Further 
analysis was executed by means of a descriptive analysis of the answers on the questionnaire, a Chi-square 
analysis and log-linear analyses (table 1). 

The Chi-square analysis of two-dimensional tables was executed to evaluate the independency of each 
specific student type variable from the large set of dependent variables as measvu-ed by specific (sets of) 
items in the questionnaire (1. prior educational level, 2. the degree of success in studying at a distance and 
3. the content domain in which the students study). When we report in this text 'significant differences', 
this implies that the independence hypothesis is to be rejected. In reporting, we will then especially focus 
on those cells in the table that are particularly illustrative to base our conclusions in relation to the 
hypotheses put forward. 

Next a log-linear analysis of the potential combined impact of two student type variables (previoxis 
educational experience and study success) was done to explain the data in the three-dimensional table 
(main effects and interaction effects); the analysis focused on a test of independence of the three variables 
in the analysis and took account of the first two student type variables as independent variables and the 
specific answer on a question as the dependent variable; 

Finally, a hierarchical log-linear analysis explored what kind of model could best fit and significantly 
explain the data in the three-dimensional table. This final model is expected to be the simplest model 
explaining still the highest proportion of variance. The results of the analysis will only be xised if the final 
model is different from the saturated model. 
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Table 1: Overview of Chi-Square analysis and log-linear analyses 
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4 Research results 

In this section, the results are discussed following the six clusters of hypotheses as discussed in part 3.1. In 
relation to each cluster, the way students responded to sets of ruestionnaire items are reviewed. At each 
level, descriptive statistics and the results of the Chi-squarc analysis arc used. Results from the log-linear 
analysis and the hierarchical log-linear analysis are discussed separately and in view of the specific 
objectives of these analyses. In appendix I the reader can find the questionnaire with an overview of the 
student responses. The proportions reported in the tables are those resulting from the weighting 
procedure. 

4.1 Prior knowledge state tests 

4.1.1 Students have no clear view on their prior knowledge. 

In relation to this hypothesis four items have been incorporated in the questionnaire: 

a Did earlier Ou-courses you studied, build on wrong expectations about your prior knowledge 

level? (variable pkall). 

The results reveal that in the perception of most students (83.2%), the Ou-courses in general do not build 
on inaccurate expectations about ones prior knowledge level. This picture is not different for specific 
student types when taking account of prior educational level or the content domain they study. But, 
w.hen comparing students with different degrees of study success in distance education, a significant 
dihcrence emerges (x^-42.829 p.OOOO). It seems that especially the group of students who did not already 
obtain a Ou-certificate or only two certificates (51.3% of the respondents) indicates that they have no 
problem (90.8% to 90.2%). There is no independency between these variables. When students are more 
successful (3 to 4 certificates or more Ou-certiftcates) the proportion of students indicating they have no 
problem with prior knowledge drops to 79.1% and 73.4%. 

b The course you actually study, does it build on prior knowledge you don't possess ? (variable - 
pkthis). 

The results reveal that in the perception of most students (88.2%), the Ou-course they are actually 
studying docs not build on inaccurate expectations about ones prior knowledge level. 
A lest of independence with each specific student type variable reveals that there might be an interaction 
with the content domain students study (x^-23.872 p.0006). The proportion of students indicating they 
have prior knowledge problems is extremely different when comparing natural science and business 
scien4:c students (23.5% compared to 8.4%). 

c - What part of the course you actually study did you already master, taking account of your 

prior knowledge, when you started studying? (variable namc^ pkbegin) 

- The impression about your initial mastery, based on your prior knowledge, could change when 
you started to study the course in detail. What part of the course did you finally estimate to 
master when you initiated your study of the actual course you have taken? (variable name^ 
pkafter) 

In this section we discuss the responses to two questionnaire items. Table 2 shows the proportions of the 
course the students estimate they already master as part of their prior knowledge at two different 
moments of the study process: before and during studying.The estimations of prior knowledge levels does 
not seem to differ depending on the two different moments. The following table repeats this information 
but reports more accurate data: 
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Table 2. Estimation of prior knowledge before and during study 

0-9% 10-24% 25-49% 50-74% 75*89% 90-100% 
before the study 33.8% 36.1% 20J% 6.3% 1.3% 0.2% 
during the study 34.9% 38,2% 19.1% 6.4% 1.2% 0.2% 

When comparing the data in table 1, we hardly perceive differences. A x^-test confirms that there are no 
significant differences in proportions (x^ ■•.2354). 

The comparison and the lack of differences questions the competence of the students to estimate their 
own prior knowledge state level. Next to comparing the outcomes at two estimation moments, one can 
ask whether the percentages in the table are not in conflict with the results obtained fron> earlier 
questions (section 4.1.1 b). For instance, up'to 88.2% of the students indicate that while studying a 
specific course, the courses do not build on wrong expectations about one's prior knowledge. Once 
more these findings support earlier research results by Dochy (1992) sustaining the generally accepted 
statement that students are not able to estimate their own prior knowledge level (Falchikov & Boud, 
1989). 

4.1.2 Students do not need prior knowledge state tests 

a I would like to get the opportunity to measure my prior knowledge state level before starting a 

Ou-course (variable - wanttest). 
This first question, helping to answer the above statement, already presents interesting results. At first 
sight, the proportions of students agreeing, disagreeing or reacting neutrally to the question are not 
extremely different: 35.5% of the students explicitly indicate they wish to know more about their prior 
knowledge state; 33.7% express a neutral attitude and 30.8% express a negative attitude. But, when we 
explore potential interrelations with student type variables, we discover significant differences for each 
of the three variables : educational level (x^- 11.563 p.0030); study success (x^- 117.37 p.OOOO); study 
domain 40.285 p.OOOl). 

Some striking differences were found. The number of students expressing a neutral or positive attitude 
is higher when the students have no higher educational background. The number of students in favour 
of prior knowledge state tests is extremely higher when we compare students without a' Ou-certificate 
(44.0%), only up to two (32.9%) and more than 4 certificates (16.2%)» Moreover, 50.2% of the latter 
group explicitly state that they do not want such tests. 

Also, remarkable differences were found in student reactions depending on the study domain of 
students. Especially students studying natural (50.1%) and social (40.6%) sciences, express a need for 
prior knowledge state tests. Students studying law (35.5%) and economical (35.1%) express especially a 
negative attitude. 

¥ 

b How much time would you be willing to invest in prior knowledge state testing? (variable - 

pktime). 

Students studying in a distance education setting are sensitive to their time management and 
interferences in their time schedal'', Th^ time constraints they have to cope with in their particular 
study setting are not to be underestimated. It is therefore relevant and necessary to ask students about 
their willingness to invest time in services offered by the institution. 

The latter is already obvious when we see that up to 29.4% of the students indicate right from the 
beginning that they do not want to spend time on solving a prior knowledge state test. This is in 
congruence with the proportion of students indicating they are not interested (30.8%) in prior 
knowledge state tests in the former section. For the further analyses we excluded this subgroup of 
students from the analysis. 

The results show that the willingness to invest time is in general high if the test would last less than an 
hour and a half (59.6%). However, this pattern differs significantly if we take account of the student 
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type variables. The proportion of students without a higher education experience that is willing to 
invest even more than half an hour is significantly higher (19.5% compared to 10.9 %). The inverse is 
also true, students with a higher education background prefer to invqst less time ( 66.3% compared to 
60%). This is shown in Figure 6. 
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Figure 6: Time investment in prior knowledge state tests; interaction with educational level 

Also the degree of study success influences the pattern in student responses. If we compare students 
without a certificate with those with more than 4 certificates the pattern differences are evident. Among 
tl»e former group the highest percentage is willing to spend between 30 and 90 minutes (67.1%). Among 
the latter 78.8% of the students are ready .to invest up to 60 minutes for a test. This suggests that 
'successful* students express a will to manage more strictly their available study time. 

If we compare students according to the content domain they study, meaningful differences can be 
observed. It is for instance interesting to compare economics and natural science students in Figure 7. 
The highest proportion (40%) of economics students only wants to invest up to 30 minutes. The highest 
proportions of natural science students prefer an investment between 30 and 60 minutes. Striking is also 
that 28% of these students are willing to invest more than 90 minutes. 
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Figure 7: Time investment in prior knowledge state tests; interaction with study domains 

c The information obtained during prior knowledge state testing could be represented with 'prior 

knowledge profiles* that reveal your weaknesses and strengths. What kind of use would you 
make of this information? 

From the results of prior knowledge state tests, as stated earlier, profiles can be derived that reveal the 
weak and strong points in a students' prior knowledge. The general question above introduced, contains 
a list of ways the information from knowledge profiles could be used. In a way, the responses to this set 
of 'information uses' are helpful to determine whether the informative value of knowledge profiles can 
be considered as specific or as general. 

Nine 'information uses' were put forward in the questionnaire. These nine can be clustered into three 
particular sets ; 

set 1: to direct content choices in relation to the study process (What do I already master 

about a course? What subparts of the course will I study more profoundly?); 

set 2: to direct planning behaviour in relation to the study process (What course will I 

choose? How will I plan my study? What will be my study route? I will plan 
accordingly my examination in an earlier phase. I will study in a more rigid way.); 

set 3: to ask for help, support (For what specific subparts will I call for help? To direct my 

demands for support.). 

Figure 8 gives a general overview of the proportions of students in favour of these three sets of 
information use. 
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Figure 8: Informative value of prior knowledge profiles; % acceptance of three clusters 

Asking students to react to the use of prior knowledge profiles to direct content choices (the first set) 
we observe high proportions of students preferring this type of information use: 

- What do I already master of a course (variable - in/what)} (51.4%) 

- What subparts of the course will I study more profoundly (variable « in/topics)} (42.0%) 

In relation to the first question "What do I already master of a course?", the answer pattern is 
significantly different considering all student type variables. Students with a higher education 
background, students with no Ou-certificate and students in specific content domains (business, 
economical, natural, social and technical sciences) especially stress this type of knowledge profile use. 
Considering the second qUestion "What subparts of the course will I study more profoundly?", the 
independency hypothesis is also rejected for the variables study success and study domain. There are for 
instance high proportions of students without a certificate (47.3%) and students studying natural sciences 
(55.8%) in favour of this information use. 

The use of knowledge profiles to plan the study (second set) is also acknowledged by the students, but 
with varying degrees of enthusiasm (tubel 3). 



Table 3: The use of knowledge profiles to plan the study 



What course will I choose (variable - infchoice)} 
How will I plan my study (variable - infplan)} 
What will be my study route (variable « infroute)} 
I will plan accordingly my examination in an earlier 
phase (variable - infexam), 

I will study in a more rigid way (variable - in/rigid). 



48.0% 
40.9% 
37.5% 

28.0% 
26.5% 



What is striking in this overview is that from the moment the type of information use affects a certain 
level of freedom and independence of the student, the students do no longer agree, From table 1 we can 
also derive that there are significant interactions with student type variables, The degree of success in 
studying interacts e.g. with the first question "What course will I study?". Students without a certificate 
agree with this type (55.2%) of information use, Students with more than 4 certificates strongly disagree 
with this kind of information use (71.6%). 

The student type variable 'content domain* influences the response patterns for 4 of the five questions. 
We summarize this in table 4, This table shows for each subqucstion the two groups with the highest 
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Table 4: Content domains and planning based on knowledge profiles 

pro 

What course will I choose? T:419% L:418% E:5Z2% N:60. 7% 

How will I plan my study? L:316% C:316% 3:46.1% N:^7A% 

I will plan accordingly my ' » 

examination in an earlier phase, C:22% B:23.8% B32% T:32,6% 

I will study in a more rigid way C:18,5% N:21J% E:32% T:312% 

T- Technical Sciences; N- Natural Sciences; L- Law; E- Economics; C- Cultural Sciences; B- Business 
Studies; S- Social Sciences 

The use of knowledge profiles as a base to ask support/help (third set^ is less preferred by the students. 
Only smaller proportions of student favour this kind of knowledge profile use: 

Table 5: The use of knowledge profiles to ask for support/help 

For what specific subparts will I call for help (variable - in/help)} 25.4% help 1 

To direct my demands for support (variable - in/support), 25.6% help 2 

Moreover, and here there are striking differences, in relation to all student type variables there are 
significant interactions. Students with a higher educational background are e.g. less enthusiastic for the 
information use 'help' (18.0%) or 'support' (20.1%), compared to students with a lower educational 
background: 'help' (32.9%), 'support' (31.9%). 

The proportions of students in favour of these kinds of knowledge profile use are also significantly 
different when we consider the independent variable 'study success'. We compare e.g. the proportions of 
students who respond positively according to the number of Ou-certificates the students already gained. 
The more certificates acquired, the less enthusiasm can be perceived for help- and/or support-functions. 



Table 6: The use of knowledge profiles to ask for help for different students 

0 cert, U2 cert. 3*4 cert, >4 cert, 
asking for help 29, 5% 25, 1% 17, 7% 16, 1% 

asking for support 28,4% 24,9% 18,2% 21,4% 

Considering the third independent variable 'content domain', there are also striking differences in 
proportions in favour or against this kind of knowledge profile use. If we simply consider the data in 
the next table we see that consistently the same content domains display the highest pro reactions: 
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Table 7: The use of knowledge profiles to ask for help in different domains 
pro 

asking for help B:17.2% T:23% E:35.4% S:36.4% 

asking for support , B:17% 7:26. 1%S:3L 7% E:36.6% 

T*- Technical Sciences; N» Natural Sciences; L» Law; Em Economics; C» Cultural Sciences; B» Business 
Studies'; S^- Social Sciences 



d Personalised and written feedback in relation to self-tests should be a valuable option, (variable 

- selftest) 

The potential of prior knowledge profiles is that studentf can get personal or written feedback. 63.8% of 
the students indicate they would appreciate this kind of feedback. Only 11.5% is not interested in this 
provision. Further analysis reveals that certain student type variables influence the response pattern of 
the students. The number of students depreciating this provision is for instance higher when the 
students have a higher education background (15.7% compared to 6.5%). 

The same pattern can be found if we compare students having as yet no Ou-certificate (6.0%) and 
students who already obtained more than 4 certificates. 28.3% of the latter group does not like this 
extra provision. Considering the content domain the students are studying, there are significant 
differences in the proportions of students agreeing or disagreeing, but the overall number of students 
showing a positive attitude is for all content domains more than 57.6% with a maximum of 68.7% 
(social sciences). 

e When taking examination at the Ou, I learn very little from the feedback or information in 

relation to my errors* (variable - exam). 
This item is not immediately linked to the topic 'prior knowledge state tests', but - in our perception - 
results in relation with this item bring further evidence for a testing practice, resulting in more relevant 
informative feedback. Moreover, the topic is linked to the 'feedback'-issue tackled in the former section. 
These students could agree, disagree or react neutrally to the following statement: 
"I learn little from my final examination. I get too little feedback or information in relation to wrong 
answers or solutions." The students react in the following way (N-1839) (table 8). 

Table 8: Learning very little from the feedback on examinations 

/ disagree 16.8% 
neutral 44 J% 
I agree 38.4% 

From Table 1 we can derive that student reactions interact with all student type variables (xO- 
For instance, students without a higher education background (41.5% agree, 13.8% disagree) express in a 
more explicit way their agreement than students with a higher education background (36.9% agree, 
18.8% disagree). Study success as expected interacts extremely with students opinion about the feedback 
value of current Ou-examination. 
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Table 9: Appreciation of the current feedback practice ; interaction with the number of certificates 
number of certificates % agreement % disagreement 



0 


23.6 


12.8 


1-2 


58.1 


16.3 


3-4 


34.8 


22.9 


>4 


51.2 


26.3 



Also the interaction with the content domain the students study is significant; cf. Figure 22. Disagreement 
is the highest within students studying technical (24.9%), economical (21.0%) and law sciences (20.0%). 
Agreement is high within students studying law (44.5%), social (43.5%) and business sciences (40.7%). 

4.1.3 The perceived and expected effects of prior knowledge measurement on study variables are limited 
a Prior knowledge has an effect on 'selectivity' in study behaviour. 

Students could, in relation to this hypothesis, react upon two questionnaire items. The first item builds on 
the condition where students themselves estimate their own prior knowledge level. The second item builds 
on information that could be provided to the students as a result of prior knowledge state tests and prior 
knowledge profiles. 

Tr.ble 10: Prior knowledge and selefctivity in study behaviour 

/ agree neutral I disagree 
I concentrate on course parts that are new to me 39 J% 39.7% 21 A% 

(variable - pkselect) 
I would select ^ourse parts to be studied more 46A% 30,8% 218%' 
thoroughly for the final examination 

(variable - choice) 

The percentages suggest thai prior knowledge measurement could influence selectivity in study behaviour. 
In relation to the first statement, student reactions interact with two independent variables ^educational 
level' and 'study domain*. The proportion of students concentrating on new parts as a consequence of 
estimated prior knowledge state levels, is higher when the students have a higher education background. 
Especially students studying economical sciences (51, 9%) and technical sciences (51, 7%) agree with this 
statement. 

But reactions to the second statement are also interrelated with student variables. Striking is for instance 
the difference in proportions of students agreeing when we distinguish students without certificates (50.6%) 
versus students with more than 4 certificates (34.8%). 

Natural sciences (29.5%) and business sciences students (28.5%) react most negatively, whereas the 
proportions of students with a positive attitude are the highest among technical sciences (48.2%) social 
sciences (47.4%) and law sciences (47.5%) students. 

b The information acquired from prior knowledge state tests can influence study time management. 

Two questions in the questionnaire asked whether students would change their study speed when they got 
information about their prior knowledge state level. 
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Table 11: Prior knowledge state tests and study time management 

/ agree neutral I disagree 
I would take examimtion earlier 34.1% 39.6% 26.3% 

(variable - quicker) 

Yes No 

I would plan in a more rigid way my study 26.5% 73.5% 

(variable - in/rigid) 

The data in the-table are interesting. At first sight a contradiction in the response behaviour of the student 
is suggested by the very different proportions of students reacting negatively. But, both statements ask 
students to react to different aspects of study time management: 

the first statement leaves all autonomy to the student to determine the moment to take an examination. 
A certain proportion of students would be willing to do this (34.1%). 

The second statement asks for an attitude towards a more drastic, long-term aspect of time management. 
Here we perceive an extremely negative attitude towards such kind of time management planning. 73.5% 
of the students reject this option. 

The data in the table get even more interesting when we research significant interactions with student type 
variables. Study success and study domain influence responses to the first statement. Experienced students 
with more than 4 certificates (33.9%) react e.g. more negatively than students without a certificate (24.5%). 

In contrast, the proportions switch when we look at the proportions of students in favour of the option 
offered: the proportion of experienced students drops to 24.2% but the proportion of students without 
certificate raises to 36.6%. Students studying social sciences (37.3%) and technical sciences (43.2%) are more 
positive towards this type of time management whereas law students (31.7%) and economics students 
(42.1%) express the highest proportions of negative attitudes. 

In relation to the second statement there is only a significant interaction with the study domain: cultural 
sciences (81.5%) students and natural sciences (79.3%) students react much more negatively. Technical 
sciences (32.3%) and economics sciences (30.2%) students react more positively. 

c. Considering the potential of the technological evolution, it is possible to present individual students 

a specific printed version of a course. Such an individualised print-out, matched to my prior 
knowledge and experience, is a good option, (variable name^ print) 

If recent technological developments were used to derive from prior knowledge measurement a tailored 
course* by printing an adapted version of the course, students do not express an extremely negative (36.5%), 
neutral (33.0%) or positive attitude (30.5%). 

But this global picture changes when we perceive significant interrelations between response behaviour and 
all the student type variables (table 12). 
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Table 12: Students* perceptions on using inidividual printed courses matched to their prior 
knowledge 



No higher education background 
Higher education background 

No certificate 
> 4 certificates 



I disagree I agK^e 
316% 34.3% 
417% 26*4% 



317% 
45.9% 



3yi% 

21.0% 



Business sciences 
Natural sciences 



417% 26.8% 
32.4% 37.0% 



The first two variables indicate that experienced students reject more readily this possibility. 

d. Students with a good portion of prior knowledge in a certain content domain should get the 
opportunity to follow a suited study route, (variable name^ route) 

At this moment students, taking courses from a diploma line have to take a standard number and/or 
elaborated set of courses. When asked to react to this statement, we can measure to what extent students 
wish to get adapted study routes (e.g. shorter). 




Figure 9: Prior knowledge state* and study route; interaction with study domain 

The proportion of students in favour of this possibility is very high (62.4%). And in line with expectations 
there are significant interactions with student type variables: study success and study domain. Although 
more than 50% of all students is in favour of this option, we see that students without a certificate show 
rather extreme positive reactions (66.0%) than students with more than 4 certificates (50.5%). 

When controlling for the domain the stunents study, as shown in figure 9, we perceive an extremely 
positive attitude of students studying natural sciences (70.5%), social sciences (69.2%) and technical sciences 
(69.3%). 
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e. When studying my course, I need a tool to determine - on my own - how good I already master 
the course-content, (variable name^ mastery) 

72,5 % of the students accept this statement. Only 77% reject it. This attitude clearly indicates the need 
of students to have more information about their study progress. 

The significant interactions in student responses with student type variables do not change this picture. 
They only reveal that e.g. students with no higher education background (76.8%) do express this need to 
a higher extent than those with a higher education background (68.4%). 

A comparable interaction is perceived when controlling for the number of certificates: students without 
certificate (76.3%) are more positive towards the statement than students with more than 4 certificates 
(60.9%). 




Figure 10: Prior knowledge tests: a tool to control mastery of the study process; interaction with 

study domains 

The significant interaction with study domain is especially due to the different proportions in favour of the 
statement among natural sciences students (79.6%) and economics sciences students (65.5%). 



4.1.4 The perceived effect of PKST on guidance, support is limited 

Three questions in the questionnaire measured the extent the students were willing to accept guidance or 
support based on the results of prior knowledge state measurement. We summarize the results in table 13. 
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Table 13: Perceived effect of prior knowledge state tests 
No Yes 

L I would use infcrmation on my prior knowledge 

to direct my demand for support (variable - in/support). 74.4% 25.6% 

2. I would use information on my prior knowledge 

to ask for help from a student tutor (variable - injhelp). 74.6% 25.4% 

I disagree Neutral I agree 

3. I would use the information to ask advise from 

experts in relation to the parts I have to study 35,6% 32.1% 32.3% 
(variable - advise) 

Making use of a third party to direct one's own study does not seem to be an attractive option to most 
students. The high percentages of 'no' reaction support this conclusion. 

From Table 1 we learn that the way students react is also significantly influenced by the three independent 
variables: educational level, study success and study domain. The following 2 figures depict these 
interactions. 




Figure 11: Prior knowledge and help/support; interaction with student type variables 
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Figure 12: Prior knowledge stale tests; advise from experts, interaction with student type variables ' 

For example, the proportion of students in favour of the three statements is always higher among students 
without a higher education background than among students with this background. Secondly, when 
comparing students without a certificate and those with more than four such certificates the same consistent 
picture can be perceived: the proportion of students without a certificate in favour of the statement is 
always higher than the proportion of students with more than 4 certificates in favour of the idea. Thirdly, 
checking for the variable 'study domain', reveals that the proportions of students taking courses in 
economics sciences and social sciences,which are in favour of the idea,are much higher than the group of 
favourable students who follow courses in other content domains. 



4.1,5 Log-linear results 

As stated in section 3.4, we incorporated in our analysis techniques log-linear and hierarchical log-linear 
analyses for a specific purpose. In contrast to the x^-technique we try to check to what extent two of the 
three independent variables also play a combined effect to explain the specific data-structure in the three- 
dimensional table. We repeat that: 

- our log-linear analysis focuses on a test of independence of the three variables in the analysis and takes 
account of the first two student type variables as independent variables and the specific answer on a 
questionnaire item as the dependent variable; 

- the hierarchical log-linear analysis explores what kind of model can explain the data in the three- 
dimensional table. This model is expected to be the simplest model explaining still the highest proportion 
of variance. 

Table 1 summarizes the results of these analyses (G^ and p- values). We focus in this section only on the 
analyses in relation to prior knowledge state tests. The test of independence of the three variables in the 
model is highly informative. In relation to most dependent variables about prior knowledge tests we 
perceive that the independency hypothesis is to be rejected (p <.01). This confirms the results of the two- 
dimensional x^'^^sts. Example: analysis of the results for the question whether students want prior 
knowledge state tests (variable^ wanttest) reveal a very significant G^-value (p-5.E-21). We can read in the 
three first columns, the very significant results of the chi-square analyses; each independent variable interacts 
with student response behaviour. The log-linear analysis adds to this that it might be possible that a 
combined effect of the two independent variables *study success' and 'educational level' influences the 
response pattern. Very significant p-values are indicative of this possible phenomenon. The hierarchical log- 
linear analysis confirms this • to a high extent • since the final model contains the interaction between 
wanttest x educational level and the interaction between wanttest x study success. This finding is a first 
indication that the independent variables 'educational level' and 'study success' are useful operational 
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definitions of the variable ^student type*. Of high importance is the fact that the interaction of both 
independent variables (educational level x study success) is not a part of the final model. 

The table also shows three examples of models containing third*order interactions: the variables pkthis, pkall 
and pkafter. In these three cases, the final model contains also the third-order interaction between the two 
independent variables (IV) and IVj) and the dependent variable (DV) &nd all second-order interactions and 
main effects of the single variables: 

(IV, X IVj X DV) + (IV, X DV) + (IVj X DV) + (IV, x IV^) + IV, + IV^ + DV. 

This finding does indicate that for these three dependent variables the impact of each single independent 
variable or their interaction with the DV is not enough to account for high proportions of the variance. 
But, it is remarkable that these three dependent variables are linked to the first questions in the 
questionnaire. This could suggest that students were still orienting themselves towards the topic 'prior 
knowledge state tests' when responding to these items. The impact of background variables could be 
overruled by this 'orientation behaviour'. 

In relation to three variables the independency hypo'hesis is not rejected for the log-linear analysis: 
in/plan, in/exam and infrigid. Here we perceive in the final model, after a hierarchical log-linear analysis, 
that only main effects of the variables in the model are retained to account for the highest proportion 
of variance. The single effect of each independent variable is always retained in the final model, thus 
giving extra support to our earlier statement that these IV can be considered as good operational 
definitions of the general variable 'student type'. 

In general, the results of the log-linear and hierarchical log-linear analysis confirm the results obtained 
from the x^-analyses. But they add an extra dimension to these results by indicating that the discrete 
independent variables 'educational level' and 'study success' are each but part of a complex set of 
possible operationalisations of the concept 'student type'. Two-way interactions and even three-way 
interactions also play a part. But, taking account of the high level of agreement between the x^-res\ilts 
and the results form the log-linear analyses, the results indicate that the two independent variables in 
our models ar; already useful and relevant operational conceptions of the complex variable 'student 
type'. 



4.2 Progress tests do not answer specific student needs 

This hypothesis about progress tests has been split up into two subhypotheses: 

4.2.1 The possibility for students to determine study progress by making use of progress tests is an 
option to be offered, (variable name^ wish) 

Table 14: Using progress tests 

/ agree 753% 
I disagree 7,8% 
Neutral 16,9% 

The results in the table 14 hardly need further comments. This could be predicted from the way 
students reacted to the statement in section c.5. There is also a significant interaction with each of the 
three student type variables. The difference between the proportion of student in favour of progress 
tests with higher education (70.4%) and without higher education (79.5%) is already remarkable. But, if 
nrr^ compare proportions of students in favour of progress tests without a certificate (83.0%) and with 
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more th^^n 4 cenificates (55.2%) the difference becomes even more striking. 

Thirdly, the content domain the students study also plays a role: Business sciences students (10.3%) and 
law sciences students (10.6%) are much more against the idea of progress tests than students studying 
other domains, whereas the proportion of students in favour of the idea and studying natural sciences 
(88.6%) is up to 10% higher than students in other domains. 

4.2.2 To get regularly the possibility to take progress tests would be helpful to structure my study in 
time and to take examination much earlier, (variable name^ regular) 

Only one question of the questionnaire evaluated the potential impact of progress test measurement on 
study behaviour. With 55.7% pro and 17.1 % against the idea of making use of progress tests to adapt 
ones study time management, progress seem to answer a certain need of students. 
That this *need* is different between students can be derived from the chi-square results. Students 
without higher education background (79.5%) are more positive towards the idea than students with a 
higher education background (70.4%). K we compare proportions of students in favour of but without a 
certificate (67.6%) or with more than 4 certificates (33.0%) the difference becomes even more striking. 

Students taking courses in cultural sciences (58.1%) and social sciences (58.7%) are to a higher extent in 
favour of time management based on progress tests than students studying other domains. 

4.2.3 Log-linear results 

The discussion presented in 4.1.5 can be repeated in relation to student opinions and reactions in 
relation to progress tests: 

Firstly, the results support the findings of the x^-analyses. The independency hypothesis is to be 
rejected. There is an interaction between the student responses to progress tests and the two 
independent variables 'educational level* and 'study success*. 

Secondly, the analyses confirm that the independent variables are useful operational definitions of the 
general variable 'student type*. Both variables wish, and regular, in interaction with the independent 
variables, are retained in the final model after hierarchical log-linear analysis. No three-way interactions 
are observed. 



4.3 Study contracts do not answer specific student needs. 

As stated in section 3.1, study contracts do not represent an alternative evaluative format. We have 
incorporated this subject as a logical extension to the alternative evaluation formats *prior knowledge 
state tests* and *progress tests* ii^ order to get extra information on the students* views about the 
consequences they are willing to attach to the results of new evaluation formats. Actually, at the Dutch 
Open university, there is a vivid interest in study contracts as a way to strengthen the institute-student 
link, assuming a better and higher degree of students taking follow-up courses (Van der Velden, et al., 
1992). Part of the potential content of a study contract could be related to (formal) use of rvaluacion 
results. 

4.3.1 After getting info about my prior knowledge and advices based on it, I would be prepared to 
discuss with the to make a *contract* about my study route and study planning, (variable 
name contract) 

32.7 % of the students accept the idea of *study contracts*, 45.47o reject the idea and 22.0% express a 
neutral attitude. Acceptance or rejectance is significandy influenced by the three student type variables. 
Figure 40 depicts the impact of 'educational level* and 'study success*. The negative attitude is for 
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instance much highe** among students with a higher education background (52.5%) than among students 
without this educational experience (39.0%). 

Also the variable study success plays a comparable role: students without certificates are much more in 
favour (35.9%) or less against (41.5%) than those with already more than 4 certificates (29.3% in favour, 
53.5% against). 

As to the variable 'study domain', students studying business sciences (52.5%) and economical sciences 
(49.0%) are much more against study contracts than those taking courses in other domains: 

The proportion of students in favour of this potential initiative is higher among the law sciences 
(33.2%), social sciences (39.9%) and technical sciences (33.2%) compared to students studying other 
content domains. 



4.3.2 Log-linear analysis 

Again, the discussion presented in 4.1.5 can be repeated here as*we already did in section 4.2.3: 
First of all, the log-linear analysis results support the findings of the x^-analysis. The independency 
hypothesis is rejected. There is an interaction between the student responses to study contracts and the 
two independent variables 'educational level' and 'study success'. 

Second, the analysis confirms that the independent variables are useful operational definitions of the 
general variable "student type'. Both variables 'wish' and 'regular', in interaction with the independent 
variables, are retained in the final model after hierarchical log-linear analysis. No three-way interactions 
are observed. 



ERLC 



31 



Alternative Assessment Procedures 



5 Discussion 

In discussing the results reported above, we will structure the discussion according to the six sets of 
hypotheses as stated earlier. But» considering some particularities of the response behaviour of the 
students, we start the discussion with some general reflections. 

5.1 Responding to the questionnaire: a learning process 

when examining the construction of our questionnaire we stressed complicating issues that could 
interfere with the response behaviour of the students since they are asked to react on evaluation formats 
they are not acquainted with. The solution to this problem was a careful construction of the 
questionnaire by gradually presenting to the students very specific potential implications/consequences 
of alternative evaluation formats. This brings about that the stud<:ats gradually are introduced to a fuller 
picture of these alternative evaluation approaches and that their initial responses to questions can be - 
from this point of view - different from those at the end of the questionnaire. We could, if the latter is 
true, for instance expect that the students take more and more extreme positions towards the itenr-s 
presented at the beginning or end of the questionnaire. A quick glance at the response patterns in the 
tables of appendix I shows that this is not the case. The opinions vary, the proportions of neutral, 
positive or negative responses switch from item to item. Moreover, and this will be illustrated in the 
next sections, the response pattern is rather consistent when comparable implications/consequences are 
presented to the students. 

Nevertheless, a certain 'learning process* could yet be observed when analyzing the students responses. 
We already referred to it in dealing with the log-linear results about prior knowledge state tests. There 
we found three-way interactions between the dependent and independent variables to be able to account 
for variance in the three-dimensional matrix, and this in relation to some initial items in the 
questionnaire. We tried to explain this by suggesting that the students were still orienting themselves 
towards the topic *prior knowledge state tests* when responding to these items. The impact of 
background variables could be overruled by this 'orientation behaviour*. 

5.2 Prior knowledge state tests 

Four main hypotheses were put forward in relation to prior knowledge state testing. The results of each 
of these will be discussed in relation tc the theoretical background and earlier empirical findings. 

5.2.1 Students have no clear view of their prior knowledge level. 

The information in the results section, based on the reactions of student to three items, is helpful to 
accept or reject this hypothesis. 

First of all it is remarkable that in general, students indicate no major problems in relation to courses 
wich would build on wrong expectations about their prior knowledge. This could suggest that students 
are not aware of mismatches between course content and their prior experiences/knowledge. 
But this is countered by various tests of independence with independent variables. We perceive for 
instance a very logical and coherent pattern in the way students who already acquired more certificates 
react to these items. These more experienced students take second-level and/or third level courses and 
enter domains for which they have less prior knowledge. As a result it is logical that they mention 
higher degrees of mismatches between their prior knowledge state level and the course content 
elaboration. 

Also interesting in this context is the fact that students taking courses in study domains that build on 
very explicit prior knowledge mention proportionally higher degrees of prior knowledge problems; a 
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typical example is the study domain natural sciences. The coherence and nature of the findings discussed 
here give us not enough grounds to reject this first hypothesis in relation to prior knowledge e tests. 
The fact that past research has shown that students are not able to rate their own level of prior 
knowledge (Falchikow & Boud, 1989; Wagemans, et al., 1991; Dochy, 1992) can be sustained. Moreover, 
an important finding is that general aspects such as the domain of study provides interesting 
information. This investigation shows several times that prior knowledge and its use is domain-specific 
(Alexander, 1992; Dochy, 1992). 

5.2.2 Students do not need prior knowledge state tests. 

I'ive items in the questionnaire were presented to the students to be able to test this hypothesis. Prior 
knowledge testing was presented a: affecting the following study behaviour and study process aspects: 

- uncertainty about one mastery when starting a course 

- time management 

- the way information obtained from prior knowledge profiles is to be used 

- the need for personalised and written feedback 

- the lack of feedback in current evaluation practices 

Generally, it can be concluded that there is a need for prior knowledge state testing, i.e. 69.2% of all 
respondents don't have any objections. Interesting is that this need is concentrated within certain 
subsamples, being students with lower educational background and students without and with no more 
than two certificates. Also interesting is that the nature of the domain plays a role in this: for example 
50% of all students in natural sciences is in favour. 

Striking is that most students in favour of prior knowledge testing do prefer to spend considerable time 
on this: one hour and a half. This time management should be seen in the light of prior educational 
experience: students with higher education experience do want to spend less, those without higher 
education experience want to spend more time. Also, the less certificates a students has, the more time 
he wants to invest. 

Concerning the use of information obtained from knowledge profiles, three types of use were selected: 
to direct content choices, to plan the study process, to ask for help and support. 

Half of the students do want to know what they already master from a particular course. Students with 
a higher education background, students with no certificate and students studying business studies, 
economics, natural, social and technical sciences especially stress this type of knowledge profile use. 
Knowledge profile use in order to plan the study is not that popular, except for choosing courses (48%) 
and for students with no certificates (55.2%). 

Knowledge profile use as a base for support is also particularly requested by students with no higher 
education experience and without any certificates. 

An advantage of the use of knowledge profiles is that students receive personalised and written 
feedback, a service that is highly welcomed by students. Only 11.5% of the students is not interested in 
this provision, being students with a higher education background and with several certificates. Students 
generally find that there is a lack of feedback when talking about the current examinations. The more 
exams a student takes, the more need for specific feedback. 

Apart from the view and utility on the different functions of knowledge profiles, the variables that are 
of prime importance arc stressed. As shown earlier by Dochy & Sieenbakkers (1988) and Dochy (1992), 
previous educational level operationalised as having 'higher education' or 'no higher education' is a 
significant variable. Next, as earlier shown by Alexander (1990) and many others (for an overview, see 
Dochy, 1992) using prior knowledge is domain-specific. The variable 'domain' seems to be of 
importance. Finally, study success expressed as an amount of certificates is significant as found earlier by 
Powell et al. (1990). 
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5.2.3 The perceived and expected effects of PKST on study variables are limited. 

The results show evidence to reject this hypothesis. Related to our second statement related to 
information that could be provided to students as a result of PKST, 77.2 % of the students say not io 
disagree with the fact that they would select parts of the course to be studied more thoroughly. This 
would even be used more by students with a higher education background. 

Related to their time managcmem, as expected on the earlier stated literature, students do not want an 
intervention in their long-term time-management. However, when the autonomy is left to the student, 
he might use a provision (such as PKST) in order to take an examination earlier (73.7%). Also this fact 
differs for domains and amount of certificates. 

Finally, a tailored course matched to one's prior knowledge is acceptable for 69.5% of the students., 
being mainly students with no higher education background and no certificates yet. 
Students are also in favour of specific study-routes when having high prior knowledge. 

5.2.4 The perceived effect of PKST on guidance, support is limited. 

This hypothesis can be confirmed. A larger proportion of the sample does not express a positive answer 
to these questions. This is also in line with earlier research (Powell, Dochy) which shows that involving 
*a third party* for support is not well appreciated by students. The '25% up to 32 % of the sample in 
favour of a third person for guidance are clearly also those students having no higher education 
experience and having no certificates up till now. 

These results are consistent with earlier research. Boon et al (1990, p.31) discovered e.g. that 70% of a 
sample of students (N-1123) prefer the actual educational system with much freedom or even more 
freedom instead of a system with restrictive aspects. Important in their results was the fact that students 
sometimes expect more provisions, but objected against shortcutting of their freedom as a consequence 
of these provisions (p. 30). 

S.3 Progress tests and study contracts 

a Progress tests do not answer specific student needs. 

In answer to the question if students need a tool to determine themselves how good they master the 
content, 92.3% replies that it is the case. This clearly shows an extremely positive attitude towards 
progress tests and confirms the findings of earlier research as given in the introductory part. The few 
students that do not want this are in the camp of higher education background and more than 4 
t eriifiiiUes. A second, but direct question concerning the use of progress tests reveals exactly the same 
results (7.8% against). Differences between several groups are rather small, except for differences in 
domain (deviation of 10%). 

Apart from this positive result, with 55.7% pro and only 17.1% against, the idea of using progress tests 
to adapt ones study time management seels to answer a real need of students. 
Since progress is shown to be one of the major problems in distance education (e.g. Powell, 1991; 
l)nihy, 1992), the implementation of progress tests could probably provide a solution. 

b Study contracts do not answer specific student needs. 

Since it is dear that students cannot estimate their own prior knowledge level, the fact that students do 
not like intei-ventions in their time-management from a third party (this investigation) and are not able 
to state their own study program intentions (Powell, 1991), it is expected that this hypothesis cannot be 
rejected. This seems to be the case. In congruence with the finding of Van dcr Velden et al.(1992) that 
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only 16% of the students is interested in study contracts, we found 32.7% of the students accept the 
idea, but 45.4% reject it. Striking is that students in business sciences and economics are even more 
against study contracts. 

6 Conclusions 

This investigation stresses the findings of earlier research and shows that development of concrete 
instruments can lead to a considerable use by the students. Overall, one can say that there is a realistic 
need for prior knowledge and progress testing and that thivS will lead to better study results and a higher 
through-flow of students. Prerequisite is that the correct variables are taken into account. 

Students are not able to estimate their own entry level, however there is a need for assessment of prior 
knowledge and a translation into knowledge profiles. For a majority of the students the individual and 
specific feedback from knowledge profiles is highly necessary and students are willing to invest 
considerable time. This information will be used for making content choices, for planing the study 
process and as a basis for support. Hence, several variables have shown to be very useful for 
differentiating between groups and between functions of knowledge profiles. These variables are: 
previoxjs educational level (higher or no higher education), domain and study success (amount of 
certificates). These results are confirmed and corroborated in our log-linear analyses. 

The study shows that students do not want any interference in their time-management (e.g. by tutors) 
but providing them with tools to monitor their entry-level and progress could lead to a faster study- 
route. Students do favour new tools as tailored courses and study-routes, adapted to individiial 
knowledge profiles. Extremely high is the expressed need for a tool such as progress tests. These have 
also a much better chance for influencing the pace of studying, rather than study contracts which are 
much less acceptable for students. 

This investigation shows that the development of alternative assessment procedures will be better 
accepted as a basis for study-support (as shown in the earlier teaching-learning model) than traditional 
human support. Moreover, the different functions it can serve for different groups is larger, and when 
taking account of the main variables large groups of students can get a better through-flow. 
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